MEMORANDUM September 13, 2018 


TO: Mechiel Rozas 
Secondary Literacy Director 
FROM: Carla Stevens 


Assistant Superintendent, Research and Accountability 


SUBJECT: LITERACY IN THE MIDDLE (LIM): IMPLEMENTATION AND EFFECTS ON MIDDLE 


SCHOOL LITERACY PERFORMANCE AND GROWTH, 2017-2018 


This report highlights the implementation of Literacy in the Middle (LIM) and its effect on reading and 
math performance and growth of middle school students. LIM is a districtwide initiative that uses read 
aloud, small group instruction, independent reading, and writing instruction to teach literacy skills 
across the curriculum. The implementation involved summer and year-long professional development 
for teachers. A teacher survey, repeated-measures design, regression, and descriptive analyses 
were used to evaluate the implementation and effects of LIM. 


Key findings include: 


Teacher survey respondents appeared to agree that the LIM professional development and 
curriculum framework and support were beneficial for implementing LIM in their classrooms and 
that they were able to use key strategies (read aloud, small group instruction) and resources in 
the process. 

Teacher respondents appeared to show less consensus in their agreements that their students 
could independently select and read “Just Right” books during daily independent reading in their 
classrooms. 

Teacher respondents tended to somewhat agree or disagree that they used the Renaissance 360 
universal screener to assess and monitor student reading performance and growth or to place 
students in flexible instructional groups. 

A key concern for survey respondents was how to effectively incorporate LIM into core course 
content areas like science and math. 

LIM had small to large effects on the mean STAAR 3-8 reading and math performance of sixth, 
seventh, and eighth-grade students, respectively. The differences in the pre (2017) and post 
(2018) LIM performance were statistically significant. 

Being identified as at-risk for school dropout was the strongest predictor of STAAR 3-8 reading at 
the sixth, seventh, and eighth grades for LIM students. The prediction was inverse, lowering the 
average reading performance on the STAAR 3-8 tests. 

Being identified as gifted and talented (G/T) was the strongest predictor of math performance at 
the sixth, seventh, and eighth grades. The prediction was positive, increasing the average math 
performance on the STAAR 3-8 tests. 


Further distribution of this report is at your discretion. Should you have any questions, please contact 
me at 713-556-6700. 
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LITERACY IN THE MIDDLE (LIM): IMPLEMENTATION AND 
EFFECTS ON MIDDLE SCHOOL LITERACY PERFORMANCE 
AND GROWTH, 2017-2018 


Executive Summary 


Literacy in the Middle (LIM) is a sixth- through eighth-grade literacy initiative being implemented in the 
Houston Independent School District (HISD). The initiative is an extension of the Literacy by 3 initiative 
being implemented at the elementary school level, district-wide. LIM, however, focuses on literacy across 
the curriculum, including the core content areas of science, math, and social studies using reading, writing, 
speaking/listening, and thinking domains. LIM uses read aloud, small group instruction, independent 
reading, and writing instruction as key strategies in its implementation. Summer and year-long professional 
development were held to prepare teachers for and to facilitate the implementation of LIM. In addition, 
teachers had access to classroom, book club, and read aloud libraries, comprehension toolkits, paper and 
electronic magazines among other resources in support of LIM implementation. 


Repeated-measures designs, a web-based teacher survey, and descriptive data analyses were used to 
evaluate the effect of LIM on sixth- through eighth-grade students’ reading and math performance on the 
Renaissance 360 universal screener and the State of Texas Assessments of Academic Readiness 
(STAAR) grades 3-8 tests. Multiple regression analyses were also used to predict LIM students reading 
and math performance based on key demographic and educational factors. 


Key findings 


e Teacher survey respondents appeared to agree that the LIM professional development and curriculum 
framework and support were beneficial for implementing LIM in their classrooms and that they were 
able to use key strategies (read aloud, small group instruction) and resources in the process. 


e Teacher respondents appeared to show less consensus in their agreements that their students could 
independently select and read “Just Right” books during daily independent reading in their classrooms. 


e Teacher respondents tended to somewhat agree or disagree that they used the Renaissance 360 
universal screener to assess and monitor student reading performance and growth or to place students 
in flexible instructional groups. 


e Akeyconcern for survey respondents was how to effectively incorporate LIM into core course content 
areas like science and math. 


e LIM had small to large effects on the mean STAAR 3-8 reading (0.21, 0.64, 0.31) and math (0.15, 0.36, 
0.92) performance of sixth, seventh, and eighth-grade students, respectively. The differences in the pre 
(2017) and post (2018) LIM performance was statistically significant (p<.0001). 


e Being identified as at-risk for school dropout was the strongest predictor of STAAR 3-8 reading at the 
sixth (0.39), seventh (0.33), and eighth (0.35) grades for LIM students. The prediction was inverse, 
lowering the average reading performance on the STAAR 3-8 tests. 
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e Being identified as gifted and talented (G/T) was the strongest predictor of math performance at the 
sixth (0.38), seventh (0.35), and eighth (0.32) grades. The prediction was positive, increasing the 
average math performance on the STAAR 3-8 tests. 


Recommendations 


e Based on teacher survey responses, every effort needs to be made to ensure that LIM teachers use 
the Renaissance 360 universal screener as a tool for assessing and monitoring students’ reading levels 
and reading growth and use the results to form flexible instruction groups. 


e Because teacher survey respondents expressed some level of uncertainty regarding students’ selection 
and use of “Just Right” books for independent reading, students and teachers may need additional 
training on book selection and in establishing more opportunities for independent reading and 
documenting reading. 


e To ensure reading and math gains are maintained, it is important to reinforce the key elements and 
practices of the initiative through continued professional development, walkthroughs, and other support 
services throughout the summer and the 2018-2019 school year. 


e Consider focusing professional development on incorporating literacy into core content areas, 
particularly science and math, like word problems, in response to these and other related concerns 
raised by respondents. This may require needs assessments of LIM teachers to better target 
professional development and improve its effectiveness in the delivery of LIM. 


e In response to requests for more resources, where possible, additional resources should be made 
available, following an audit of resource needs, to LIM teachers to improve the implementation of LIM. 


e Careful attention needs to be paid to at-risk students at all middle grades to improve the impact of the 


LIM initiative on students’ literacy and math performance. Sustained interventions could include a 
needs assessment involving focus groups to determine the academic needs of at-risk students. 
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Introduction 


Literacy and literacy performance and achievement in the Houston Independent School District (HISD) 
continue to be complex issues. The high percentage of economically-disadvantaged and English language 
learners enrolled in diverse classrooms with diverse literacy needs challenge performance expectations 
and targets. Notwithstanding, the overarching goal of HISD is for all students to read at or above grade 
level by the third-grade (HISD, 2017). In response, HISD has implemented several initiatives and programs 
at all levels of the school system to address the diverse literacy needs of learners. One such initiative is 
Literacy in the Middle (LIM). LIM is a middle school initiative and an extension of the Literacy By 3 initiative 
that is being implemented at the elementary school level commencing in the 2013-2014 school year. LIM 
aims to have all students in middle school (6"—8"" grade) reading on grade level by 2020 (HISD, 2017). 


LIM focuses on four key components of literacy: reading, writing, speaking/listening, and thinking. Students 
are instructed and given choice books, provided time to work with text, and receive individual support in 
small group settings, according to the Curriculum and Development Department webpage on the HISD 
website (HISD, 2017). Literacy in the Middle is buttressed on five key approaches to literacy instruction and 
development — read aloud, modeled thinking, small group instruction, independent reading, and writing 
instruction. Classrooms are configured to reflect these approaches and to facilitate seamless inclusion of 
the daily literacy strategies instruction in core content classes. Teachers, literacy leaders, and principals 
underwent professional development during the summer of 2017 in preparation for the implementation of 
LIM. There were also monthly follow-up professional development sessions throughout the 2017-2018 
school year. Ongoing walkthroughs have been used to monitor its implementation. One of the unique 
features of LIM is its focus on reading and literacy across the curriculum to include the core content areas 
of science, social studies, mathematics, and language arts. LIM provided resource materials for its program 
delivery. These included (1) classroom libraries in the four core foundation classes, (2) book club libraries 
in English language arts (ELA), (3) read aloud libraries, (4) comprehension toolkits for large- and small- 
group lesson support, (5) paper and electronic magazines for all four core classes, and (6) document-based 
questions (DBQ) kits for all social studies classes to support comprehension, analysis, and writing (HISD, 
2017). LIM is in its second year of full implementation. This evaluation is designed to provide information 
that would inform the implementation process using teacher surveys and analyses of student outcomes 
using both formative and summative reading and literacy assessment data. The evaluation was guided by 
the following questions: 


1. What were the perceptions and experiences of teachers who participated in the LIM professional 
development training? 


2. How did sixth through eighth-grade HISD students perform on Renaissance 360 universal screener 
at the BOY and EOY? 


3. What were the effects of LIM on the literacy levels of HISD sixth- through eighth-grade students? 


4. What educational and demographic factors best explained the variance in the literacy and math 
performance of HISD LIM students? 
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Literature Review 


Reading and literacy has focused on the early primary grades as determinants of later and proficient 
reading, but this is never automatic (International Reading Association, 2006). Research shows that some 
middle school students continue to struggle with reading and literacy, which may stem, in part, from 
teachers’ perceptions of those struggling readers, teachers’ own professional capacity to address the 
struggles, and teachers’ identities, roles, and responsibilities (Moreau, 2014; Ness, 2009). Further, research 
confirms that students have difficulty making the transition to more complex textbooks in middle and high 
schools, and that more that about 75 percent of students who exit third grade as struggling readers continue 
to be poor readers in high school (International Reading Association, 2006). Instruction in reading and 
literacy requires a shift from remediation to development in which literacy is taught as part of core content 
areas, including language arts. 


Generally, research on literacy has focused on instructional strategies like read-aloud (Marchessault & 
Larwin, 2014), multicomponent reading comprehension (Fogarty, et al., 2014), differentiation (Little, 
McCoach & Reis, 2014), oral repeated reading (Sukhram & Monda-Amaya, 2017), rather than the creation 
of classroom environments that facilitate reading. Other studies looked at students’ reading motivation 
(Troyer, 2017), reading engagement (Protacio, 2017), reading perspectives (Ramsay & Sperling, 2015), 
reading modality and passage genre (Dickens & Meisinger, 2017), and coaching (International Reading 
Association, 2006). Studies also examined below-average readers (Swanson, et al., 2017), struggling 
readers (Moreau, 2014), and teacher perceptions of poor readers (Moreau, 2014). The research which 
follows focuses on the kind of environment and actions required to facilitate reading in middle school or 
among young adolescents. 


A qualitative study surveyed 1,765 students in reading and language arts classes in 23 diverse schools in 
the mid-Atlantic and northeastern United States focusing on what makes children want to read in middle 
school classrooms (Ivey & Broaddus, 2001). The study also involved follow-up interviews with 31 students 
in three classrooms who were highly engaged in reading. Students valued independent reading and teacher 
read-alouds during instructional time. Students preferred actual reading or personal reasoning for reading 
rather than reading-related activities, and they were motivated to read based on the quality and diversity of 
reading materials (Ivey & Broaddus, 2001). Access to reading materials and the presence of diverse reading 
materials appeared to have been key factors in middle school reading (Ivey & Broaddus, 2001). 


Sanacore and Palumbo (2010) supported the premise for exposing students to appropriate and quality 
reading materials and to provide adequate time at school for reading engagement. Accordingly, “time spent 
reading books is the best predictor of measures of reading achievement, reading comprehension, 
vocabulary, and reading speed” (Anderson, Field & Wilson cited in Sanacore & Palumbo, 2010, p. 189). 
Sanacore and Palumbo (2010) recommended independent reading and providing adequate time to read, 
balancing independent reading with a variety of textual experiences, extension of in-school reading to at- 
home reading via balanced classroom libraries, promoting active reading and writing with drama-based 
activities, and guiding learners in activities that build and activate their vocabulary. 


Ivey and Broaddus (2001, p. 86) contended that “students in middle schools still need good reading 
instruction but many teachers may be unprepared or unable to provide it.” They proposed the need to move 
away from whole class instructions predicated on teacher-controlled reading materials, to providing 
students access to varied materials students like, giving students ownership of what they read, and making 
independent reading a priority. They proposed the following: (1) aligning instruction to readers rather than 
reading, (2) linking reading to writing, (3) reading aloud to students, (4) allowing students to read aloud, and 
(5) approaching reading instruction as a developmental process. 
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Developmentally, children do not begin to read at the same age. The reading process ranges from 
prekindergarten (prek) to third grade and occurs in five stages: (1) Awareness and Exploration of Reading 
Stage (typically prek), (2) Emergent Reading Stage (typically preK to early kindergarten), (3) Early Reading 
Stage (typically kindergarten to early first grade), (4) Transitional Reading Stage (typically late first to 
second grade), and (5) Fluent Reading Stage (typically third grade and higher) (National Institute of 
Professional Practice, 2018). Developmental growth, however, is uneven and is influenced by children’s 
environment and experiences. Consequently, their intellectual ability should not be associated with their 
developmental stages (National Institute of Professional Practice, 2018). 


Literacy in the middle appears to possess some of the recommended approaches to middle school reading 
and literacy, particularly, small group or differentiated instruction, speaking and listening or read-aloud, 
access to ‘choice’ books and time to read, and writing. It is expected that that approach should result in 
reading improvement where LIM is implemented with fidelity and where students are given adequate time 
to read. The amount of instructional time students needed to show improvement was uncertain. 


Method 


Evaluation Design 


This evaluation is mixed method study consisting of a repeated-measures design involving sixth- through 
eighth-grade students. Students at all three grade levels were eligible to participate in the program. 
However, teachers were expected to complete a professional development (PD) in preparation for 
implementation. Teachers self-selected into the PD programs, which made it difficult to identify a cadre of 
teachers who did not participate but with equal propensity for participation. Only students whose teachers 
completed the PD were included in the analysis of program effects as these would have had the knowledge 
and skills to implement the program with fidelity. A list of teachers who completed the PD was retrieved 
from the HISD OneSource employee training database. OneSource is an online Human Resource 
Management portal that includes managing professional development activities in HISD. Lists of 
participants were exported to a Microsoft Excel spreadsheet at the end of each year and submitted to HISD 
Research and Accountability Department and stored in the Microsoft Access database as part of its archival 
data. 


Repeated-measures design or paired samples are robust research designs involving a single group 
outcome, measured on the same dimensions but at different times that is, time 1 and time 2. Using the 
same group controls for all the extraneous variables makes it possible to make unbiased inferential and 
conclusive statements about the sample based on the outcome measure (Pallant, 2013). Students’ 2017 
and 2018 State of Texas Assessments of Academic Readiness (STAAR) grades 3-8 reading and math 
subtest scores or results were used as the outcome measures as well as Beginning-of-Year (BOY) and 
End-of-Year (EOY) reading and math assessment results from Renaissance Star 360 Universal Screener 
for sixth- through eighth-grade students. Renaissance Star 360 is a computer adaptive (CAT) 
comprehensive K-12 interim and formative assessment suite that provides data for screening and 
monitoring students’ progress and growth for making informed learning and instructional decisions. 


Data Collection 


Teachers who completed the LIM PD were matched to their students by Application Developers using 
unique identifiers (employee and student IDs) in the HISD Texas Student Data System Public Education 
Information Management System (TSDS PEIMS). A review of research on teacher professional 
development (PD) found that more than 14 hours of PD showed positive and significant effects on student 
achievement (Yoon, Duncan, Lee, Scarloss, & Shapley, 2009). Teachers who received substantial PD, an 
average of 49 hours over nine studies, can boost student achievement by about 21 percentile points (Yoon, 
Duncan, Lee, Scarloss, & Shapley, 2009). Students were linked to their STAAR scores by Application 
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Developers using the Texas Education Agency (TEA) STAAR 3-8 database using students’ local IDs. 
STAAR first test administration data were used in the analyses. Students required legitimate before or pre 
(2017) and after or post (2018) scores in reading and math to be included in the analyses. STAAR 3-8 
social studies, science and writing are administered only once in middle school (seventh or eighth grade) 
and could not be included in the study since pre-scores for the sample groups were unavailable. Students’ 
BOY and EOY scores on the universal screener were analyzed to determine reading and math gains. These 
were unmatched and provided a measure of middle school performance in literacy and math since LIM is 
a districtwide program. 


An online survey to capture teacher perceptions of their experiences with LIM was administered. Teachers 
who completed the LIM professional development in the core curriculum areas were the targets of this 
survey. Targeted teachers completed between 3 and 24 credit hours of PD related to LIM. Two Teacher 
Development Specialists (TDS) who worked closely with the implementation of LIM assisted with the 
evaluation planning. They were actively involved in developing and piloting the survey instruments. Their 
involvement was critical to ensure that salient aspects of LIM were included in the evaluation to determine 
the extent to which the program objectives were met. The survey was emailed to 347 respondents using 
SurveyMonkey™, a web-based survey platform with data analysis capability. A reminder was sent out 15 
days later to all participants. A total of 71 teachers, or approximately 20.5 percent responded to the survey. 
The University of Texas at Austin Center for Teaching and Learning’s analysis of survey response rates 
found an average response rate of 30% for online surveys (Saldivar, 2012). Other researchers found similar 
average response rates (Hamilton as cited in Saldivar, 2012). An online survey designed to measure 
teacher perceptions and experiences with HISD Literacy By 3 resulted in a response rate of 21.8% (HISD, 
2015). 


Data Analysis 


Paired samples t-tests were used to determine differences in students’ performance in literacy/reading and 
math after a year’s exposure to LIM and the impact of LIM on these differences using effect sizes. 
Regression analyses were also used to determine the extent to which key factors explained the variance 
in students’ average performance. Renaissance 360 BOY and EOY middle school data were also analyzed 
to measure reading gains. Analyses of survey data were conducted in SurveyMonkey™ and were 
summarized as percentages and as weighted averages where Likert-type questions were used. Data were 
presented in tables and charts. 


Limitations 


e Since all students were expected to participate in LIM as a districtwide initiative, getting a control group 
for more robust analysis and to determine program effects was not feasible. Therefore, a repeated- 
measures design was used to overcome the limitation using students’ 2017 test scores as pre-scores 
and 2018 test scores as post scores on the STAAR 3-8 tests. 


e There were school, teacher, and non-school factors that may have impacted students’ performance. 
This study did not consider these factors where students’ background characteristics were used to 
control for students’ effects. 


e Principals and teachers did not respond to repeated requests to conduct site visits, walkthroughs, or 


student focus-group discussions to gather additional data on the implementation of LIM and the 
experiences and perceptions of students. However, there were consultations with LIM staff. 
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e The analysis did not consider other middle school programs that could have been implemented 
concurrently and that may have impacted students’ reading. Nonetheless, most of these programs 
tended to have the same objectives as they were designed to improve students’ reading performance. 


Results 


What were the perceptions and experiences of teachers who participated in the LIM professional 
development training? 


Results of the survey on teachers’ perceptions and experiences with LIM are displayed in Figure 1 — Figure 
8. Figure 1 shows the composition of the teacher respondents in the survey. Details are on Table 1, 
Appendix A, p. 28. 


Figure 1. Percentage of LIM Teachers by Grades Taught During 2017-2018 School Year 
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e Most respondents (37.1%) were sixth-grade teachers, followed by eighth (30%), and seventh-grade 
teachers (27.1%). 


e About 5.7 percent of teacher respondents taught a combination of sixth through eighth grades. 


Figure 2 shows survey respondents’ class configurations for the delivery of LIM. Details are on Table 2, 
Appendix A, p. 28. 


Figure 2. Teaching Configuration of HISD LIM Classrooms, 2017-2018 
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e Most (92.9%) of respondents’ classrooms were departmentalized and 7.1 percent were self-contained. 


Figure 3 depicts the distribution of respondents by their years of teaching experience. Details are on Table 
3, Appendix A, p. 28. 


Figure 3. The Distribution of LIM Teacher Respondents by Years of Teaching Experience, 2017- 
2018 
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Teaching Experience 


e Of the 70 LIM respondents, more than one in four (27.1%) and one in five (22.9%) had between 3-5 
and 20+ years, respectfully, of teaching experience. 


e According to Figure 3, about 14.3 percent of respondents had 0-2 years of teaching experience, while 
15.7 percent had 11-15 years of teaching experience. 


e Overall, most teacher respondents (58.6%) had more than five years of teaching experience, while 85.7 
percent had three or more years of teaching experience. 


Figure 4 displays the core content areas that LIM respondents teach. Details are on Table 4, Appendix A, 
p. 28. 


Figure 4. Distribution of HISD Teacher Respondents by LIM Core Content Area, 2017-2018 
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Core Content Teaching Area 
e Most respondents were English language arts (ELA) (40.6%) and mathematics (33.3%) teachers. 
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e According to Figure 4, about 17.4 percent of LIM survey respondents were science teachers and 8.7 
percent were social studies teachers. 


Figure 5 shows the available instructional time by respondents within which LIM strategies were taught. 
Details are on Table 5, Appendix A, p. 29. 


Figure 5. Instructional Time Allocated to Daily Literacy Instruction, HISD, 2017-2018 
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Instructional time 


e More than half of LIM respondents (51.4%) had less than 50 minutes dedicated to daily literacy 
instruction, 18.6 percent dedicated 75 minutes, and 20.0 percent dedicated 90 minutes to instruction. 


e Most of the classes (90%) taught by survey respondents dedicated 90 minutes or less to literacy 
instruction. 


e About 8.6 percent of respondents dedicated more than 135 minutes to daily literacy instruction. 
LIM Professional Development and Support 


Figure 6 and Figure 7 display respondents’ degree of agreement or disagreement with their participation 
in LIM professional development activities. Table 6 and Table 7, Appendix A, p. 29 provide details. 


Figure 6. Respondents’ Attendance at LIM Professional Development Activities, 2017-2018 
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e Of the 51 teachers who responded to the Figure 7 statement, 96.1 percent attended the LIM training 
involving small group instruction, independent reading, read aloud, and writing instruction. 
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Figure 7. Respondents’ Agreement or Disagreement with Participation Statements in LIM 
Professional Activities, 2017-2018 
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Professional Development 


e Most respondents agreed that information learned during the four-day LIM training (average rating of 
3.46 out of 5.0) prepared them to deliver LIM in their classrooms. 


e According to Figure 7, most respondents somewhat agreed or agreed that information learned from the 
three-day LIM training (average rating of 3.47 out of 5.0) prepared them to deliver LIM in their 
classrooms. 


e Most respondents somewhat agreed (3.08 of 5.0 rating) that LIM leaders provided valuable support 
and information about best literacy practices. 


e Most respondents appeared to have completed face-to-face rather than the online training modules as 
indicated by the average rating of 1.91 out of possible rating of 5.0, mostly disagreeing that they 
completed the online modules. 


Teacher Efficacy and Instructional Practices 


A ten-question weighted scale (1-5) was used to measure efficacy and instructional practices teachers 
employed during implementation of LIM. Respondents were asked to state their degree of agreement or 
disagreement with related statements. The results are presented in Figure 8. Table 8, Appendix A, p. 30 
provides details. 
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Figure 8. LIM Teacher Efficacy and Instructional Practices, HISD, 2017-2018 
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Teacher efficacy and Instructional Practices 


Most teacher respondents reported to agree or somewhat agree (weighted average of 3.68 of 5.0) that, 
compared to a year ago, they had a deeper understanding of what it means to be an effective literacy 
teacher. 


Most teacher respondents agreed or somewhat agreed that digital and print reading materials and 
resources available to their classrooms were useful (weighted average of 3.76 of 5.0). Slightly less 
agreed or somewhat agreed that the materials and resources were of a high quality (weighted average 
of 3.49 of 5.0). 


Most LIM teacher respondents agreed or somewhat agreed that they facilitated read aloud activities in 
their classrooms daily (weighted average of 3.40 of 5.0). Slightly more teachers agreed, somewhat 
agreed, or strongly agreed that they facilitated small group instructions in their classrooms (weighted 
average of 3.47 of 5.0). 


Most respondents somewhat agreed or agreed that students independently selected (3.09 of 5.0 rating) 
“Just Right” text. Most respondents somewhat agreed or agreed that students read (3.06 of 5.0 rating) 
“Just Right” text during their daily independent reading in their classrooms. 


Most teacher respondents somewhat agreed or disagreed that they used the universal screener to form 
flexible instructional groups (2.74 of 5.0 rating); somewhat agreed, disagreed, or strongly agreed to 
assess students’ reading levels (2.81 of 5.0 rating); and disagreed or agreed to monitor students’ 
reading growth (2.85 of 5.0). 


Figure 9 outlines survey respondents’ perceptions and experiences with key curriculum frameworks and 
structure in their implementation of LIM. Table 9, Appendix A, p. 30 provides details. 
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Figure 9. Respondents’ Perceptions of HISD Curriculum Framework and Support for LIM, 2017-2018 
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Curriculum Framework and Support 


e Most respondents somewhat agreed or agreed (weighted average 3.46 of 5.0) that the structure and 
framework of the HISD secondary curriculum planning guide facilitated ease of planning. 


e Most respondents somewhat agreed or agreed (weighted average of 3.19 of 5.0) that the Reader’s 
Workshop Model provided structure for reading instruction in their classrooms. 


e Most respondents somewhat agreed or strongly agreed (weighted average 3.40 of 5.0) that the 
instructional strategies outlined in the HISD curriculum planning guides provide support for their 
instructional planning and delivery. 


e Most respondents somewhat agreed or agreed (weighted average 3.52 of 5.0) that the anchor charts 
outlined in the HISD master courses provided support for my instructional planning and delivery. 


Respondents were also required to provide additional comments that they may have had on the 
implementation of LIM and any additional details/information they would like to see in the HISD master 
courses to further support LIM implementation. Those who responded to the question on the master course 
spoke about adapting LIM more closely to core content courses. The additional details and information that 
were offered included: 


Providing “articles that can be used in science to assist with reading.” 
“Possibly, how to blend active/interactive components with the math literacy.” 
“More writing prompts for middle school math” and “something targeted towards math.” 


One respondent added “Please, actually, discuss and practice reading strategies in science classroom and 
not just what a science teacher should already be doing like ISNs and labs. The idea of LIM should be 
adding to our pedagogy to be interdisciplinary and increasing literacy, not how to be a science teacher.” 


Another respondent noted, “Literacy in the traditional sense is hard to implement in mathematics. Hopefully, 
we can move to shortened, effective ways to analyze and breakdown word problems.” 
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Respondents also wanted to see “more higher order level thinking questions in the master course lessons 
that challenge students to use critical thinking skills.” Respondents also made pitches for “more resources,” 
“informational text support,” “better content materials,” “technology support for resources in Title 1 schools,” 
“more practice lessons for students,” and for “units that could be easily adapted for reading different novels 
along with the units.” 


How did sixth through eighth-grade HISD students perform on Renaissance 360 universal screener 
at the BOY and EOY? 


Renaissance 360 Universal Screener 


The Beginning-of-Year (BOY) and End-of-Year (EOY) literacy and math results on the Renaissance 360 
were used to determine changes in the performance of students who may have been exposed to the LIM 
initiative. All middle school students’ BOY and EOY unmatched scores were used. Figure 10 and Figure 
11 show the overall percentage of LIM students who met the benchmark (2 40" percentile) on the 
Renaissance 360 literacy and math assessments, respectively. 


Figure 10. Percentage of HISD Students by Ethnicity Performing atior Above the 40" Percentile on 
the Renaissance 360 Reading Assessment, 2017-2018 
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Source: Renaissance 360 Universal Screener, HISD Microsoft Access database. 
e According to Figure 10, about 30 percent of LIM students in sixth through eighth grades performed at/or 


above the 40" percentile on the Renaissance 360 universal screener at the BOY and EOY in reading. 


e There were negligible increases in the percentage of LIM sixth- through eighth-grade students who 
performed at/or above the 40" percentiles on the Renaissance 360 universal screener at the BOY and 
EOY in reading. 
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Figure 11. Percentage of HISD Students by Grade Performing at/or Above the 40" Percentile on the 
Renaissance 360 Math Assessment, 2017-2018 
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Source: Renaissance 360 Universal Screener, HISD Microsoft Access database. 


e More than half of all sixth- through eighth-grade students met the math BOY and EOY benchmarks (2 
40" percentile) on the Renaissance 360 universal screener. 


e There were slight increases in the percentage of sixth- (0.5%), seventh- (2.9%), and eighth- (1.9%) 
grade students who met the BOY and EOY math benchmarks (2 40" percentile) on the Renaissance 
360 universal screener. 


Figure 12 and Figure 13 show the proportion of students by race and ethnicity who performed at or above 
the BOY and EOY literacy and math benchmarks (2 40" percentile) on the Renaissance 360 universal 


screener at the sixth, seventh, and eighth grades. 


Figure 12. Percentage of HISD Students by Race/Ethnicity Performing at/or Above the 40" Percentile 


on the Renaissance 360 Reading Assessment, 2017-2018 
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Source: Renaissance 360 Universal Screener, HISD Microsoft Access database. 
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e There were increases in the percentage of students in three of the four major ethnic groups who met 
the reading benchmark from BOY to EOY on the Renaissance 360 universal screener at the sixth and 
seventh grades. The percentage of seventh-grade Asian students who met the benchmark remained 
unchanged. 


e For eighth-grade students, increases in reading from BOY to EOY were evidenced for Black and 
Hispanic students. 


e A majority of Asian and White students met the BOY and EOY reading benchmark on Renaissance 
360 universal screener at the sixth, seventh, and eighth grades. Most Hispanic students met the BOY 
and EOY benchmarks at the sixth and eighth grades and the EOY at the seventh grade. 


e Less than 50 percent of Black students met the BOY and EOY reading benchmark on Renaissance 
360 universal screener at the sixth, seventh, and eighth grades. 


Figure 13. Percentage of HISD Students by Ethnicity Performing at/or Above the 40" Percentile on 
the Renaissance 360 Math Assessment, 2017-2018 
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Source: Renaissance 360 Universal Screener, HISD Microsoft Access database. 


e A majority of Asian (between 66.3 and 77.8%) and White (between 72.0 and 76.3 percent) students 
performed at or above the BOY and EOY math benchmarks on the Renaissance 360 universal screener 
at the sixth, seventh, and eighth grades. 


e Between 22.1 and 26.3 percent of Black and between 22.5 and 25.7 percent of Hispanic students 
performed at or above the BOY and EOY benchmark on the Renaissance 360 universal screener at 
the sixth, seventh, and eighth grades. 


e Black students showed decreases in the proportion who met benchmark on the Renaissance 360 math 
assessment between the BOY and EOY at all three grade levels. There were increases for seventh- 
and eighth-grade Hispanic students. 


Figure 14 and Figure 15 display the percentage of students by economic status who performed at or above 


the reading and math benchmarks (2 40" percentile), respectively, on the Renaissance 360 universal 
screener at the sixth, seventh, and eighth grades. 
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Figure 14. Percentage of HISD Students by Economic Status Performing atlor Above the 40‘ 
Percentile on the Renaissance 360 Reading Assessment, 2017-2018 
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Source: Renaissance 360 Universal Screener, HISD Microsoft Access database. 


There were increases from the BOY to the EOY in the percentage of non-economically-disadvantaged 
students who performed at or above the 40* percentile on the Renaissance 360 universal screener in 
sixth-, seventh-, and eighth-grade reading. 


There were slight increases from the BOY to the EOY in the percentage of economically-disadvantaged 
students who performed at or above the reading benchmark at the seventh and eighth grades. 


Non-economically-disadvantaged middle-school students outperformed their economically- 
disadvantaged peers on the Renaissance 360 reading universal screener at all three grades. 


Figure 15. Percentage of HISD Students by Economic Status Performing atlor Above the 40‘ 


Percentile on the Renaissance 360 Math Assessment, 2017-2018 
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A higher percentage of non-economically-disadvantaged students (68.4 — 75.5%) compared to 
economically-disadvantaged students (46.6 — 51.1%) performed at or above the BOY and EOY math 
benchmarks on the Renaissance 360 universal screener at the sixth, seventh, and eighth grades. 


HISD Research and Accountability 16 


LITERACY IN THE MIDDLE REPORT, 2017-2018 


e Both groups showed either increases or no changes in the year-end percentages of LIM students who 
met benchmark (2 40" percentile) on the Renaissance 360 universal screener at the sixth, seventh, 
and eighth grades. 


e Non-economically-disadvantaged middle school students outperformed their economically- 
disadvantaged peers on the Renaissance 360 math universal screener at all three grades at the BOY 
and EOY. 


Figure 16 and Figure 17 show the percentage of students by home language who performed at or above 
the reading and math benchmarks (2 40" percentile), respectively, on the Renaissance 360 universal 
screener at the sixth, seventh, and eighth grades. 


Figure 16. Percentage of HISD Students by Home Language Performing atior Above the 40% 
Percentile on the Renaissance 360 Reading Assessment, 2017-2018 
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Source: Renaissance 360 Universal Screener, HISD Microsoft Access database. (Data only). 


e A higher percentage of students with “Other” home languages that were neither Spanish nor English 
performed at or above the BOY and EOY literacy benchmarks (2 40" percentile) on the Renaissance 
360 universal screener in reading at the sixth, seventh, and eighth grades compared to students with 
English and Spanish home languages. There was a decrease, between the BOY and EOY, in the 
percentage of students with “Other” home languages who performed at or above benchmarks for all 
three middle-school grades. The number of “Other” home language students in the sample ranged from 
555-518 for the three grades. 


e The percentage of students with English or Spanish home languages who performed at or above the 
literacy benchmark (2 40" percentile) on the Renaissance 360 reading assessments increased from 
the BOY to EOY for the sixth, seventh, and eighth grades. 
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Figure 17. Percentage of HISD Students by Home Language Performing at/or Above the 40‘ 
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e Except for the sixth-grade students with Spanish home language, there were increases between the 
BOY and EOY in the percentage of LIM students who performed at or above the math benchmarks, on 
the Renaissance 360 universal screener at the sixth, seventh, and eighth grades. 


e Seventh-grade students with home languages that were neither English nor Spanish had the largest 
increase in the proportion of students (31.9%) who performed at or above the math benchmarks 
between the BOY and EOY on the Renaissance 360 universal screener. 


e The proportion of eighth-grade students with English or Spanish home languages who met the 
Renaissance 360 math benchmark increased over the 2017-2018 school year. 
State of Texas Assessments of Academic Readiness (STAAR) 


The results which follow are based on the repeated-measures design using STAAR 3-8 exam results of 
students whose teachers completed LIM professional development. Table 10, Appendix A, p. 31 shows the 
composition of students who make of the sample. The district demographic composition is given for 
comparative purposes. Based on Table 10: 


e The percentage of at-risk students had the largest sample size variation at 9.4 percentage points among 
the sixth- through eighth-grade samples. 

e The smallest sample size variation was among Asian students at 0.8 percentage points for the sixth- 
through eighth-grade samples. 

STAAR Reading 


Figure 18 and Figure 19 display the comparative percentage of LIM students who met Approaches, Meets, 
and Masters standards on the 2017 and 2018 STAAR 3-8 reading and math tests. The 2017 scores are 
considered pre-scores and the 2018 scores are considered post-scores. 
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Figure 18. Comparative Percentage of LIM Students Who Met Approaches, Meets, and Masters 
Standard on the 2017 and 2018 STAAR 3-8 Reading Test 
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e The percentage of LIM sixth-grade students whose teachers completed the LIM professional 
development and who met Approaches Grade Level standard on the 2018 STAAR 3-8 reading test 
decreased by 4.6 percentage points. The percentage of these students who achieved Meets and 
Masters standards decreased by 8.3 and 6.7 percentage points, respectively. 


e The percentage of seventh-grade students whose teachers completed the LIM professional 
development and who met Approaches Grade Level standard on the 2018 STAAR 3-8 reading 
test increased by 9.1 percentage points. The percentage of these students who achieved Meets 
and Masters standards increased 10.0 and 9.2 percentage points, respectively. 


e The percentage of eighth-grade students whose teachers completed the LIM professional 
development and who met Approaches Grade Level standard on the 2018 STAAR 3-8 reading test 
increased by 5.4 percentage points. The percentage of these students who achieved Meets and 
Masters standards increased by 4.2 and 3.6 percentage points, respectively. 
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STAAR Math 


Figure 19. Comparative Percentage of LIM Students Who Met Approaches, Meets, and Masters 
Standard on the 2017 and 2018 STAAR 3-8 Math Test 
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e The percentage of LIM sixth-grade students who met Approaches Grade Level standard on the 
2018 STAAR math test decreased by 5.2 percentage points. The percentage of these students 
who attained Meets and Masters standards decreased by 5.7 and 5.6 percentage points, 
respectively. 


e The percentage of seventh-grade students whose teachers completed LIM professional 
development and met Approaches Grade Level standard on the 2018 STAAR 3-8 math test 
decreased by 3.1 percentage points. The percentage of these students who attained Meets and 
Masters standards decreased 3.2 and 0.9 percentage points, respectively. 


e The percentage of eighth-grade students whose teachers completed LIM professional development 
and met Approaches Grade Level standard on the 2018 STAR 3-8 math increased by 12.0 
percentage points. The percentage of these students who attained Meets and Masters standards 
increased by 5.8 and decreased by 0.4 percentage points, respectively. 


What were the effects of LIM on the literacy levels of HISD sixth- through eighth-grade students? 


Figure 20 and Figure 21 show the comparative STAAR mean reading scale scores by grade, using paired 
t-tests, for students who had 2017 (pre) and 2018 (post) test scores and whose teachers completed LIM 
professional development. First STAAR test administration scores were used in the analyses. Table 11 and 
Table 12, Appendix A, p. 31 provide details. The effect sizes were calculated using Hedge’s g formula and 
interpreted using Cohen? (1988) effect size designations and the U.S. Department of Education IES 
substantially important effect size of 0.25. Cohen’s d and Hedge’s g are interpreted in a similar way. 


1 Cohen (1998) designated effect sizes as small = 0.2, medium 0.5, and large as 0.8 
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Figure 20. Comparative (Paired T-Test) STAAR 3-8 Mean Reading Scale Score for LIM Sixth- 
Through Eighth-Grade Students in the Sample, 2017 and 2018 
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e Sixth-grade students in the sample had a higher mean reading scale score one year after exposure to 
LIM (M = 1554.1, SD = 140.5) in 2018 compared to their mean reading scale score the previous (m = 
1524.2, SD = 148.6). The difference was statistically significant, t (8,229) = 28.5. p<.0001 (two-tailed). 
The Hedge’s g statistic (0.21) indicate a small effect size (Table 11, Appendix A, p. 31). 


e Seventh-grade students in the sample had a higher mean scale score one year after exposure to LIM 
(M = 1616.4, SD = 133.5) in 2018 compared to their mean reading scale score the previous year, 2017 
(M = 1525.6, SD = 135.0). The difference was statistically significant, t (10,423) = 95.4, p<.0001. The 
Hedge’s g statistic (0.64) indicated a medium and substantially important effect size (Table 11, 
Appendix A, p. 31). 


e Eighth-grade students in the sample had a higher mean reading scale score one year after exposure 
to LIM in 2018 (M = 1649.5, SD = 140.0) compared to their mean reading scale score in 2017 (M = 
1605.7, SD = 145.1). The difference was statistically significant, t (10,669) = 51.3, p<.0001. The Hedges 
g statistic (0.31) indicated small but substantially important effect size (Table 11, Appendix A, p. 31). 


Figure 21. Comparative (Paired T-Test) STAAR 3-8 Mean Math Scale Score for LIM Sixth- to Eight- 
Grade Students in the Sample, 2017 and 2018 
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e Figure 20 shows that LIM sixth-grade students in the sample had a higher mean scale score (M = 
1628.3, SD = 139.4) on the 2018 STAAR 3-8 math test compared to their mean scale score on the 
2017 STAAR 3-8 reading test (M = 1606.4, SD = 152.1). The difference was statistically significant, t 
(8,220) = 19.8, p<.0001. The Hedge’s g statistic (0.15) indicated a small effect size (Table12, Appendix 
A, p. 24). 


e Seventh-grade LIM students in the sample had a higher mean scale score (M = 1630.2, SD = 133.5) 
on the 2018 STAR math test compared to their mean scale score on the 2017 STAAR 3-8 math test 
(M = 1584.0, SD = 135.0). The difference was statistically significant, t (9,862) = 52.1, p<.0001. The 
Hedge’s g statistic (0.36) indicated a medium and substantially important effect size (Table 12, 
Appendix A, p. 24). 


e Eighth-grade LIM students in the sample had a higher mean scale score (M = 1680.7, SD = 148.8) on 
the 2018 STAAR 3-8 math test compared to their mean scale score on the 2017 STAAR 3-8 math test 
(M = 1536.6, SD = 165.0). The difference was statistically significant, t (7,981) = 54.5, p<.0001. The 
Hedge’s g statistic (0.92) indicated a large and substantially important effect size (Table 12, Appendix 
A, p. 24). 


What educational and demographic factors best explained the variance in the literacy and math 
performance of HISD LIM students? 


Results of the regression analyses designed to identify predictors of performance for students whose 
teachers completed LIM professional development and implemented LIM in their classrooms are presented 
in Table 13 and Table 14 (Appendix A, pp. 32, 33) for reading and math, respectively. The standardized 
coefficient or Beta are provided in the findings results statements, which follow. 


STAAR 3-8 Reading 


e Overall, the regression model for sixth-grade predicted 48 percent of the LIM students’ scale scores on 
the 2018 STAAR 3-8 reading test. It predicted 43 percent of the reading performance for the seventh 
grade and 47 percent for the eighth-grade LIM students. 


e All five predictors used in the regression models (economically disadvantaged, at risk?, special 
education, Limited English proficiency (LEP), and gifted and talented (G/T) identification were 
statistically significant (p<.0001, .05) in predicting LIM students’ reading scale scores at sixth-, seventh- 
, and eighth-grades. All predictors, except G/T identification, had an inverse effect on the mean or 
constant reading scale score for sixth (1630.6), seventh (1701.1), and eighth (1749.6) grade students. 


e Being assessed as at-risk was the strongest predictor of reading performance for LIM students at sixth 
(39%), seventh (33%), and eighth (35%) grades. These students would have scored lower than the 
average scale scores. 


e =G/T identification predicted 32 percent of LIM sixth grade reading performance on the 2018 STAAR 3-— 
8 test, while it predicted 30 and 22 percent for seventh and eighth grades, respectively. This was the 
only positive predictor in the model. 


? PEIMS defined at-risk indicator as a student who is at risk for dropping out of school using state defined criteria. Such a student 
must be less under 26 years of age and may possess one or more of thirteen criteria or conditions (See Texas Education Agency, 
2018). 
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e Being economically disadvantaged was the weakest predictor of seventh- and eighth-grade LIM 
students’ performance on the 2018 STAAR 3-8 reading test. Being LEP was the weakest predictor for 
sixth-grade reading. 


STAAR 3-8 Math 


e Theregression models explained 42 percent of the variation in the 2018 STAAR 3-8 math scale scores 
for LIM sixth-grade students, 39 percent for seventh-grade, and 39 percent for eighth-grade students. 


e All five predictors used in the regression models (economically disadvantaged, at risk, special 
education, LEP, and G/T identification), except being economically disadvantaged at the eighth grade, 
were Statistically significant (p<.0001, .05) in predicting the variance in LIM students’ 2018 STAAR 3- 
8 math scale scores at sixth, seventh, and eighth grades. All predictors, except G/T identification, had 
an inverse effect on the mean or constant reading scale score for sixth (1674.9), seventh (1693.5), and 
eighth (1784.5) grade. 


e Being identified as G/T was the strongest predictor of math performance for LIM students at sixth (38%), 
seventh (35%), and eighth (32%) grade. G/T predicted above average performance on the 2018 
STAAR 3-8 math test. 


e Being identified as at-risk was the strongest adverse predictor of LIM sixth-grade math scale scores 
(35%) on the 2018 STAAR 3-8 test and predicted 32 and 30 percent of the seventh- and eighth-grade 
2018 STAAR 3-8 math performance, respectively. These had adverse effects on the average scale 
scores. 


e Being economically disadvantaged was the weakest predictor of sixth-, seventh- and eighth-grade LIM 
students’ performance on the 2018 STAAR 3-8 reading test. The results were statistically significant 
(p<.0001). Being LEP was the weakest predictor for the sixth-grade math. 


Discussion 


The purpose of this evaluation was to determine literacy growth and achievement among middle school 
students who were exposed to the LIM initiative using teacher surveys, descriptive statistics, repeated 
measures designs, and regression analyses. The evaluation showed that the program had small to large 
effects using Hedges’ g on the reading and math performance of students at the seventh and eighth-grade 
levels and substantially important effects (0.25) at the seventh and eighth grades. It showed that there were 
statistically significant increases in the mean scale scores of students on the 2018 STAAR 3-8 reading and 
math tests at all middle grades compared to their 2017 STAAR performance. This was expected since LIM 
appeared to promote the key initiatives highlighted in the literature for improving middle school literacy and 
reading performance. 


The data showed increases in the percentages of students who met or exceeded the Approaches Grade 
Level standard for seventh and eighth-grade reading, and eighth-grade math between 2017 and 2018. In 
addition, the data showed decreases in the percentages of students who met or exceeded the Approaches 
Grade Level standard in sixth-grade reading and sixth and seventh-grade math. However, results of the 
repeated measures or paired t-tests showed gains in the mean scale scores from 2017 to 2018 for all middle 
school grades in the study. Therefore, students showed improvement, indicating that the initiative was 
effective after one year but for some students, the improvement may not have been adequate to meet the 
Approaches Grade Level standard. This was the case for sixth-grade reading. Because math content 
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involves more than reading, and because the reading component could not be disaggregated, and its 
effects measured separately, one needs to be cautious with how the math results are interpreted. 


Regression analyses showed that being identified as at-risk for school dropout was the strongest predictor 
for reading and math performance at all middle grades. The findings were all statistically significant. This is 
significant because the at-risk indicator is a composite of thirteen factors that included class failure, class 
retention, pregnancy, disciplinary issues, homelessness, displacement, and prior dropout. Students are 
considered at risk because of one or more of these factors. These factors had adverse effects on middle 
school LIM students’ performance in reading and math. G/T identification was the only positive predictor 
and predicted that students were likely to have higher than average scores on the STAAR 3-8 math and 
reading scale scores. The predictions were statistically significant for all three grades. 


Site visits were not conducted to observe what happened in the classrooms with LIM implementation 
expectations and to confirm the findings of existing research on middle school literacy instruction and 
reading. Self-reported data from surveyed teachers provided some insights into their perceptions and 
experiences with the implementation of LIM. Survey responses should be interpreted with caution. The 
findings may not be representative of the entire universe of LIM teachers’ experiences and perspectives 
due to the 20.5 percent response rate. 


Teacher survey respondents generally agreed that they learned from the LIM professional development 
and were prepared for the delivery of LIM. Teachers did not appear to fully agree that they received full 
support from their campus leaders in implementing LIM. Respondents appeared to be less confident that 
their students were able to select “Just Right” books, independently (3.06 of 5.0 weighted average) or read 
“Just Right” books during their independent reading period (3.09 of 5.0 weighted average). Several 
researchers (Sanacore & Palumbo, 2010; Ivey & Broaddus, 2001) have advocated for more independent 
reading and that students select their texts. However, Ness (2009) showed that content-area teachers in 
science and social studies were unlikely to teach reading comprehension because of the pressure to cover 
content for standardized testing, felt unqualified or were not responsible to do so. 


It did not appear that teacher respondents consistently tested using the universal screener to assess 
students’ reading levels (2.81 of 5.0 weighted average), monitor their progress (2.85 of 5.0), or used the 
results to place students in flexible learning groups (2.74 of 5.0 weighted average). These strategies provide 
feedback on how to design instruction that meets students’ needs. Failure to screen students may mean 
that they may not have received the targeted assistance they may have needed to demonstrate reading 
progress. 


Results from the Renaissance 360 universal screener for all middle school grade levels appear to confirm 
this dilemma since about 30 percent of students at those grades met the reading benchmark and about 50 
percent met the math benchmark. Before and after, but unmatched reading and math Renaissance 
assessment results for all three middle-school grades, were used in the analyses. There was very little 
change in the percentage of students who met benchmark (2 40" percentile) on the Renaissance 360 
reading and math assessments for sixth, seventh, and eighth grades. 


With respect to the open-ended questions, teacher respondents offered several suggestions for improving 
implementation of LIM. Key among their concerns were the need to better incorporate literacy in the 
instruction of science, math, and other core content areas, and the need for more resources in the support 
of LIM. 


Recommendations 
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Based on teacher survey responses, every effort needs to be made to ensure that LIM teachers use 
Renaissance 360 Universal Screener as a tool for assessing and monitoring students’ reading levels, 
and reading growth, and use the results to form flexible instruction groups. 


Because teacher survey respondents expressed some level of uncertainty regarding students’ selection 
and use of “Just Right” books for independent reading, students and teachers may need additional 
training on book selection and in establishing more opportunities for independent reading and 
documenting reading. 


To ensure reading and math gains are maintained, it is important to reinforce the key elements and 
practices of the initiative through continued professional development, walkthroughs, and other support 
services throughout the summer and the 2018-2019 school year. 


Consider focusing professional development on incorporating literacy into core content areas, 
particularly science and math, like word problems, in response to these and other related concerns 
raised by respondents. This may require needs assessments of LIM teachers to better target 
professional development and improve its effectiveness in the delivery of LIM. 


In response to requests for more resources, where possible, additional resources should be made 
available, following an audit of resource needs, to LIM teachers to improve the implementation of LIM. 


Careful attention needs to be paid to at-risk students at all middle grades to improve the impact of the 


LIM initiative on students’ literacy and math performance. Sustained interventions could include a 
needs assessment involving focus groups to determine the academic needs of at-risk students. 
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Appendix A 

Table 1. Distribution of LIM Survey Respondents by Grade Level Taught, HISD, 2017-2018 

Grade Level % n 
Sixth Grade 37.1 26 
Seventh Grade 27.1 19 
Eighth Grade 30.0 21 
Other (please specify) 5.7 4 
Total 70 
No responses 1 


Table 2. Distribution of LIM Survey Respondents’ Teaching Configurations, HISD, 2017-2018 


Teaching Configurations % n 
Self-contained 7.1 5 
Departmentalized 92.9 65 
Other (please specify) 2.8 2 
Total 70 
No response 1 


Table 3. Teaching Experience of LIM Survey Respondents, HISD, 2017-2018 


Years of Teaching Experience % n 
0-2 years 14.3 10 
3-5 years 27.1 19 
6-10 years 7.1 5 
11-15 years 15.7 11 
16 -20 years 12.9 9 
20+ years 22.9 16 
Total 70 
No responses 1 


Core Content Area % n 
ELA 40.6 28 
Mathematics 33.3 23 
Social Studies 8.7 6 
Science 17.4 12 
Total 69 
No Reponses 2 
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Table 5. Distribution of LIM Instructional Time Reported by Survey Respondents, HISD, 


2017-2018 

Instructional Time % n 
Less than 50 minutes 51.4 36 
75 minutes 18.6 13 
90 minutes 20.0 14 
135 minutes 1.4 1 
More than 135 minutes 8.6 6 
Total 70 
No response 1 


Table 6. Percentage of Respondents who Attended LIM Training, HISD, 2017-2018 


Professional Development % n % n 
| attended the Literacy in the Middle training (Small-group 
Instruction, Independent Reading, Read Aloud and writing 96.1 49 3.9 2 51 
instruction). 
Total 5L 
No responses 20 


Table 7. Survey Respondents’ Perceptions and Experiences with LIM Professional Development 


and Support, HISD, 2017-2018 


Somewhat 
Agree 


Strongly 


Strongly NIA Total 
Disagree 


Agree Weighted 
Average 


Disagree 
Professional Development and Support 


% n % n % n % n % n % n n 


The information | learned during the 
four-day training prepared me to begin 
implementing Literacy in the Middle in 
my classroom. 

The information | learned during the 
three-day training prepared me to begin 
implementing Literacy in the Middle in 
the classroom. 

The Literacy in the Middle leaders on 
my campus provide valuable support 
and information about best literacy 
practices. 

| completed the online modules instead 
of the face-to-face training for Literacy 33.3 17 15.7 8 7.8 4 3.9 2 3.9 2 35.3 18 51 1.91 
in the Middle. 
Total 51 


3.9 2 13.7 7 31.4 16 19.6 10 21.6 a1 9.8 5 51 3.46 


2.0 1 13.7 7 29.4 15 27.5 14 15.7 8 11.8 6 51 3.47 


11.8 6 13.7 7 39.2 20 13.7 of 15.7 8 5.9 3 51 3.08 


No responses 20 
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Table 8. LIM Survey Respondents’ Perception of Their Efficacy and Instructional Practices, 


HISD, 2017-2018 


Teacher Efficacy and Instructional i hap Disagree lt Agree Strongly Agree Total Weighted 
7 isagree Agree 
Practices Average 
% n % n % n % n % n % 
In comparison to one year ago, | have 
a deeper understanding of what it 
means to be an effective literacy 5.7 3 3.8 2 32.1 17 34.0 18 24.5 13 53 3.68 
teacher. 
The digital and print reading materials 
and resources available to my 3.8 2 11.3 6 34.0 18 34.0 18 17.0 9 53 3.49 
classroom are high-quality. 
The digital and print materials and 
resources available to my classroom 0.0 ie} 5.9 3 33.3 17 39.2 20 21.6 14. 51 3.76 
are useful. 
| facilitate interactive read alouds ona 
daily basis in my classroom. 7.6 4 15.1 8 24.5 13 35.9 19 17.0 9 53 3.40 
| facilitate small-group instruction 3.8 2 18.9 | 10 283 15 24.5 13 | 245 13 53 3.47 
daily in my classroom. 
Students read “Just Right” texts by 
their independent levels during 
indegiencent reading tine dally inn 13.2 7 208 | 11 | 26.4 14 26.4 14 13.2 7 53 3.06 
classroom. 
Students in my classroom 
independently select “Just Right” 17.0 9 18.9 10 20.8 11 24.5 13 18.9 10 53 3.09 
texts for independent reading. 
Luse the universal: screener 10 form zo )=—g9_Ss| 23 | 15 | 264] 14 208 | 1 | 76 4 53 2.74 
flexible instructional groups. 
l use the universal screener to assess 
student reading levels. 17.0 9 24.5 13 26.4 14 24.5 13 7.6 4 53 2.81 
luse the universal screenertomonitor 479 9 | 245 | 13 | 226 | 12 233 | 15 | 76 4 53 2.85 
reading growth. 
Total 53 
No responses 18 


Table 9. LIM Survey Respondents’ Perceptions and Experiences with Curriculum Framework 


and Support, HISD, 2017-2018 


Strongly 


Curriculum Framework 


Disagree 


Somewhat 
Agree 


Agree 


Strongly 
Agree 


NIA 


and Support 


Disagree 


Total 


% n 


% 


n % 


n % 


n % 


Weighted 
Average 


The structure and 
framework of the 

HISD secondary 
curriculum planning 
guides facilitate the ease 
of planning. 


5.9 3 


7.8 


4 37.3 


19 29.4 


15 


17. 


7 9 2.0 


51. 


3.46 


The Reader’s Workshop 
Model provides structure 
for reading instruction in 
my classroom. 


9.8 5 


13.7 


t 23.5 


12 21.6 


4, 


13. 


7 7 17.7 


51 


3.19 


The instructional 
strategies outlined in the 
HISD curriculum planning 
guides provide support 
for my planning and 
delivery of instruction. 


7.8 4 


11.8 


6 35.3 


18 19.6 


10 


23: 


5 12 2.0 


51 


3.40 


The anchor charts 
outlined in the HISD 
master courses provide 
support for my planning 
and delivery of 
instruction. 


3.9 2 


7.8 


4 33.3 


17 27.5 


14 


17. 


7 9 9.8 


51 


3.52 


Total 


51 


No responses 


20 
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Table 10. Composition of LIM Samples for Analysis of STAAR Performance, 2017-2018 


Attributes Sixth Grade+_ Seventh Grade? _ Eighth Grade? District? 
n=9,210 n= 11,354 n= 11,522 n = 214,175 

Economically Disadvantaged No 20.4 18.3 19.9 24.1 
Yes 79.6 81.7 80.1 75.9 

At Risk No 35.5 27.4 26.1 28.4 
Yes 64.5 72.6 73.9 71.6 

Special Education No 91.4 91.5 92.9 92.8 
Yes 8.6 8.5 7.1 7.2 

Limited English proficiency No 67.7 68.7 74.0 68.5 
Yes 32.3 31.3 26.0 31.5 

GIT Identification No 75.9 78.4 83.0 84.3 
Yes 24.1 21.6 17.0 15.7 

Ethnicity Asian 2.9 2.2 3.0 4.1 
Black 28.7 27.3 25.0 24.0 

Hispanic 61.2 65.8 66.4 61.8 

White 6.0 3.7 4.8 8.7 


Source: ‘HISD TSDS PEIMS Microsoft Access database, 2017-201.8 
2TSDS PEIMS 2017-2018, Fall Collection, Resubmission, Texas Education Agency PDM1-120-009 v18.1.1. 


Table 11. Paired-Samples T-test and Effect of LIM on STAAR 3-8 Reading by Grade. HISD, 2017- 


2018 


Paired Differences 


7 95% Confidence Sig. (2- Effect 
Reading n Mean Std. oe Interval of the t df ot i 
Mean oe Error : tailed) Size 
Deviation Mean Difference 
Lower _ Upper 
Sixth Post =n 
Grade Readini Y 
= 8,230 ——_._ 29.9 95.3 14 27.9 320 285 8,229 000 0.21 
Recaiiy 1524.2 
Seventh Post 1616.4 
Greek —peaaing 10,424 —————._ 90.8 97.1 1.0 88.9 92.7 95.4 10,423 000 0.64 
eS 1525.6 
Reading 
Eighth Post 1649.5 
Crave —peadng _. 10,670 —————._ 43.8 88.2 0.9 42.1 45.55 51.3 10,669 000 0.31 
1605.7 
Reading 


Effect Size (Hedges’ g): Small = .2; Moderate = .5, and Large = .8; U.S. Department of Education Institute for Education Sciences (IES) Substantially Important Effect = .25 


Table 12. Paired-Samples T-test and Effect of LIM on STAAR 3-8 Math by Grade, HISD, 2017- 


2018 


Paired Differences 
ms : 
Std. 95% Confidence 


Sig. (2- Effect 


Math n Mean Std. Interval of the t df : 
Mean Deviation Error Difference tailed) Size 
Mean 
Lower Upper 
Sixth Post 
Grade Math a0283 
= — + 8.221 ———_— 21.9 100.2 1.0 19.7 24.0 19.8 8,220 0.000 0.15 
ae 1606.4 
Math 7 
a 16902 
Pre. 9,863 —————_ 46.2 88.053 88 44.5 47.9 52.1 9,862 0.000 0.35 
Math 1584.0 
Grade _ Math 1680.7 
——__ 7,982 ———_ 144.2 236.500 2.6 139.0 149.4 54.5 7,981 0.000 0.92 
Pre- 
Math 1536.6 


Effect Size (Hedges’ g): Small = .2; Moderate = .5, and Large = .8; U.S. Department of Education Institute for Education Sciences (IES) Substantially Important Effect = .25 
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Table 13. Predictors of Reading Performance for LIM Students, 2017-2018 


ede prcaieis goa Praia ty 95.0% Confidence Interval for B 
B Beta Lower Bound Upper Bound 
Sixth Constant 1630.6** 1624.7 1636.6 
Grade Economically -27.8 -0.08** 20.2 22.5 
Disadvantaged 
At Risk -114.1** -0.39** -119.4 -108.7 
Special Education -75.7** -0.15** -83.4 -68.0 
LEP -10.3** -0.03** -15.2 -5.4 
G/T Identification 106.0** 0.32** 100.4 111.5 
RR? 0.48 
F 1654.3** 
Seventh Constant 1701.1** 1694.8 1707.5 
Grade “Economically -14.8"* -0.04** 20.2 9.5 
Disadvantaged 
At Risk -107.0** -0.33** -112.4 -101.6 
Special Education -103.4** -0.20** -110.8 -96.0 
LEP -38.3** -0.12** -42.9 -33.7 
G/T Identification 104.2** 0.30** 98.7 109.8 
Re 0.43 
F 1684.7** 
Eighth Constant 1749.6** 1744.0 1755.3 
Grade Economically 5.9" -0.02* -10.7 at 
Disadvantaged 
At Risk -111.4** -0.35** -116.5 -106.3 
Special Education -109.9** -0.20** -117.3 -102.4 
LEP -84.4** -0.26** -88.8 -79.9 
G/T Identification 83.4** 0.22** 77.8 89.1 
PR? 0.47 
F 2052.7** 


Dependent Variable: STAAR 3-8 Reading Scale Score 
*p <.05. **p <. 001 
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Table 14. Predictors of Math Performance for LIM Students, HISD, 2017-2018 


Unstandardized Standardized 95.0% Confidence Interval for 
' Coefficients Coefficients B 
Grade Math Predictors a 
pper 
B Beta Lower Bound Bound 
Sixth Constant 1674.9** 1668.8 1681.1 
Grade “Economically -19.8** -.06** -25.5 -14.1 
Disadvantaged 
At Risk -101.1** -.35** -106.7 -95.5 
Special Education -41.7** -.08** -49.7 -33.7 
LEP 12.9** .04** 7.8 18.0 
G/T Identification 123.4** .38** 117.6 129.2 
R? 42 
F 1322.4** 
Seventh Constant 1693.5** 1687.3 1699.7 
Grade Economically -6.5** -02* aL? “1.4 
Disadvantaged 
At Risk -97.3** -.32** -102.6 -92.0 
Special Education -66.7** -.14** -73.7 -59.6 
LEP -11.4** -.04** -15.9 -7.0 
G/T Identification 118.6** .35** 113.0 124.2 
R? 39 
F 1346.5** 
Eighth Constant 1784.5** 1776.2 1792.9 
Grade - 
Economically -0.16 .00 -6.6 6.3 
Disadvantaged 
At Risk -120.9** -.30** -128.5 -113.2 
Special Education -105.9** -.20** -114.6 -97.3 
LEP -37.0** -.12** -42.5 -31.5 
G/T Identification 158.1** .32** 148.8 167.4 
Re .39 
F 1103.9** 


Dependent Variable: STAAR 3-8 Math Scale Score 
*p<.05. **p <. 001 
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